Data Prefetcher that Adjusts Prefetch Stream Length Based on Confidence

ABSTRACT

In an embodiment, a processor comprises a data cache and a prefetch unit coupled to the data cache. The prefetch unit is configured to identify a prefetch stream in cache misses from the data cache, and the prefetch unit is configured to issue prefetches predicted by the prefetch stream to prefetch data into the data cache. More particularly, the prefetch unit implements one or more stream engines that generate prefetches for respective prefetch streams. Each stream engine is configured to maintain limit data that indicates a number of prefetches that are permitted to be outstanding beyond a most recent demand access. The stream engine is configured to increase the limit responsive to the number of demand accesses that consume prefetched data at least equaling the limit.

BACKGROUND

1. Field of the Invention

This invention is related to the field of processors and, moreparticularly, to data prefetching mechanisms in processors.

2. Description of the Related Art

Memory latencies are now a dominant factor in modern processorperformance. As processor speeds have increased over the years, memoryspeeds have failed to keep pace. As a result, memory latencies haveincreased when measured in terms of numbers of processor clock cycles.Various strategies are being employed to mitigate the impact of theseincreased latencies. Some of these strategies include various forms ofmulti-threading (allowing other threads to continue while one threadwaits for memory), larger caches, and various forms of speculationincluding run-ahead execution and result prediction.

Data prefetching can be used to alleviate performance lost to memorylatency. Data prefetchers analyze a set of memory accesses, attemptingto predict patterns within those accesses. When a pattern is recognized,prefetches can be issued to begin retrieving data from memory ahead ofwhen the program requires it.

Prefetches can also interfere with the “real” accesses generateddirectly from instruction execution (often referred to as demand fetchesor demand accesses). Accordingly, generating accurate prefetches (i.e.prefetches that have high likelihood of being the target of demandaccesses that occur in later instruction execution) is important. Aprefetcher that generates significant numbers of inaccurate prefetchescan reduce performance.

SUMMARY

In an embodiment, a processor comprises a data cache and a prefetch unitcoupled to the data cache. The prefetch unit is configured to identify aprefetch stream in cache misses from the data cache, and the prefetchunit is configured to issue prefetches predicted by the prefetch streamto prefetch data into the data cache. The prefetch unit is configured tomaintain limit data that indicates a number of prefetches that arepermitted to be outstanding beyond a most recent demand access. Theprefetch unit is configured to increase the limit responsive to thenumber of demand accesses that consume prefetched data at least equalingthe limit.

In an embodiment, a stream engine comprises one or more registers and acontrol circuit coupled to the registers. The registers are configuredto store: a first pointer identifying a next expected demand accesscorresponding to a prefetch stream that is assigned to the streamengine; a second pointer identifying a next prefetch in the prefetchstream; and limit data indicating a limit to a number of prefetches thatare permitted beyond the first pointer. The control circuit isconfigured to generate prefetches responsive to the limit data.Responsive to demand accesses consuming at least a number of prefetchesequal to the limit, the control circuit is configured to increase thelimit.

BRIEF DESCRIPTION OF THE DRAWINGS

The following detailed description makes reference to the accompanyingdrawings, which are now briefly described.

FIG. 1 is a block diagram of one embodiment of a processor.

FIG. 2 is a block diagram of one embodiment of a core shown in FIG. 1.

FIG. 3 is a block diagram of one embodiment of a data cache and aportion of a load/store unit shown in FIG. 2.

FIG. 4 is a block diagram of one embodiment of an entry for one aprefetch learning table (PLT) shown in FIG. 3.

FIG. 5 is a flowchart illustrating operation of one embodiment of a PLTcontrol circuit shown in FIG. 3.

FIG. 6 is a flowchart illustrating operation of one embodiment of a PLTlookup illustrated in FIG. 5.

FIG. 7 is a flowchart illustration operation of one embodiment of astream engine that has been allocated a prefetch stream.

FIG. 8 is a flowchart illustrating one embodiment of determining if aprefetch is ready, as illustrated in FIG. 7.

FIG. 9 is an example of one embodiment of the operation of a streamengine prefetching a stream.

FIG. 10 is a block diagram of one embodiment of a system.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims. The headings used herein are for organizational purposes onlyand are not meant to be used to limit the scope of the description. Asused throughout this application, the word “may” is used in a permissivesense (i.e., meaning having the potential to), rather than the mandatorysense (i.e., meaning must). Similarly, the words “include”, “including”,and “includes” mean including, but not limited to.

Various units, circuits, or other components may be described as“configured to” perform a task or tasks. In such contexts, “configuredto” is a broad recitation of structure generally meaning “havingcircuitry that” performs the task or tasks during operation. As such,the unit/circuit/component can be configured to perform the task evenwhen the unit/circuit/component is not currently on. In general, thecircuitry that forms the structure corresponding to “configured to” mayinclude hardware circuits. Similarly, various units/circuits/componentsmay be described as performing a task or tasks, for convenience in thedescription. Such descriptions should be interpreted as including thephrase “configured to.” Reciting a unit/circuit/component that isconfigured to perform one or more tasks is expressly intended not toinvoke 35 U.S.C. § 112, paragraph six interpretation for thatunit/circuit/component.

DETAILED DESCRIPTION OF EMBODIMENTS

In one embodiment, a prefetch unit includes a memory, referred to as theprefetch learning table (PLT) herein, a control circuit for the PLT, andmultiple stream engines. The PLT may store data to learn about prefetchstreams. Once a prefetch stream has been detected and confirmed in thePLT, the control circuit may assign the prefetch stream to a streamengine that may be configured to generate prefetches. Thus, the learning(or detection) of prefetch streams may be decoupled from the generationof prefetches in the confirmed prefetch streams. In one embodiment,since the data in the PLT is not used to generate prefetches, arelatively simple memory may be implemented. For example, a staticrandom access memory (SRAM) with a relatively small number of ports,such as a dual ported SRAM may be used. The ports are used to read anentry in response to a cache miss and to write an entry with updateddata to track a potential prefetch stream. A simpler memory is lessexpensive to implement (e.g. in terms of integrated circuit area), andthus more entries may be supported than might otherwise be possible witha more complex memory. More accurate prefetch stream identification mayresult, in some embodiments. Additionally, when a new potential prefetchstream is identified and the control circuit overwrites an entry in thePLT, the data that is overwritten in the entry is for another potential(unconfirmed) prefetch stream. Active prefetch streams are in the streamengines, and may not be affected by the overwriting of entries in thePLT.

The stream engines may also be relatively simple, since the streamengines may not include the circuitry to learn a prefetch stream.Instead, the stream engines are issued a confirmed stream and areconfigured to generate prefetches and to control the number ofprefetches that are outstanding (e.g. issued and, at some later point,written to the data cache but not yet consumed by a demand access). Inone embodiment, for example, the stream engine may monitor the number ofdemand accesses that are consumed within the prefetch stream. Once thenumber of consumed prefetches equals the current limit for prefetchesoutstanding (thus indicating that the prefetch stream is accuratelyprefetching data that is later used, increasing a level of confidence inthe prefetch stream), the stream engine may be configured to increasethe limit. In one embodiment, the limit may be doubled each time it isincreased.

The memory in which the PLT is stored includes multiple entries, eachentry capable of tracking a different potential prefetch stream. Anymechanism may be used by the control circuit to select an entry. In oneembodiment, for example, the memory may receive the program counter (PC)address from which an instruction was fetched, where the instructioncorresponds to a memory operation that is detected to miss in the datacache. The PC is sometimes referred to as the instruction pointer (IP).The remainder of this instruction may refer to the PC, and the IP may besynonymous. Generally, the PC or IP may refer to the address from whichan instruction is fetched. The address of the memory location accessedduring execution of a load/store may be referred to as a data address(or simply an address).

At least a portion of the PC may be used as an index into the memory, toselect an entry or entries to track the potential prefetch stream. Thus,prefetch streams may be associated with instructions in this embodiment.In other embodiments, the address to the memory may be derived from thePC (e.g. a hash of the PC or a portion thereof). In embodiments thatinclude hardware support for simultaneous multithreading, the threadidentifier (ID) of the thread containing the instruction may also beused, to prevent or reduce aliasing between different threads. Forexample, multiple copies of the same application may be executing on amulti-threaded processor, and thus may have the same virtual PC eventhough the threads differ. In one embodiment, the thread ID may bematched to a thread ID field in the selected entry or entries, so thatonly an instruction from a given thread may update the data tracking thepotential prefetch stream. In one embodiment, the thread ID may also beincluded in the index generation (e.g. it may be hashed with the PC), sothat the same PC from different threads may select different entries.Such an embodiment may permit more accurate prefetch streamidentification when multiple application copies are being executed, orPCs are otherwise aliased among multiple threads. Accordingly, theaddress used to index the PLT memory may generally be used to locate anentry or entries in the memory (and may be compared to a tag in theentries, in some embodiments). The address to the PLT memory may notnecessarily be an address that can be used to access as system memory ina system with the processor.

Generally, a memory operation may be a load (memory read) or store(memory write). The operation may be an explicit load/store instruction,or may be an implicit part of an instruction that has a memory operand,in some instruction sets. A load memory operation may be more succinctlyreferred to herein as a load operation, and a store memory operation maybe more succinctly referred to as a store operation. In one embodiment,only load operations that miss the data cache may cause access to thePLT. In one implementation, software prefetch instructions may not betreated as load operations for prefetch purposes, although otherimplementations may include them.

A prefetch may generally be a memory read operation that is generated byhardware (e.g. the prefetch unit) in response to identifying a prefetchpattern among the miss addresses from a data cache. On the other hand, ademand access is a memory operation (read or write) that is directlyderived from an instruction being executed (either an explicitload/store instruction or an instruction having a memory operand, asdiscussed above). Various prefetch algorithms may be used. For example,a strided prefetch algorithm detects a pattern of addresses that areseparated by a fixed amount, referred to as the stride. That is, thedifference between consecutive addresses in the pattern is the stride.Prefetches may be generated by adding the stride to the most recentaddress and prefetching from the resulting address, adding the stride tothe resulting address to generate the next prefetch, etc. Other prefetchalgorithms are possible in other embodiments. In general, the prefetchpattern permits the addresses that may be demand accesses at some laterpoint to be predicted, and the prefetches that are predicted may beissued and the corresponding data written to the data cache. Thesubsequent demand access may be a cache hit, and thus may be lowerlatency than if prefetching had not been performed. A prefetch streammay be a collection of prefetches that correspond to a detected prefetchpattern. That is, the prefetch stream may be the predicted set of futuredemand accesses based on the prefetch pattern.

If a prefetch is correct, a subsequent demand access occurs thataccesses the prefetched data. The demand access accessing the prefetcheddata is referred to as the demand access “consuming” the prefetcheddata.

The present description refers to accesses that hit/miss the cache.Generally, an access to data that is found in the cache at the time ofthe access is a hit, and an access to data that is not found in thecache at the time of the access is a miss. The description may alsorefer to an address hitting/missing in other circuitry such as a missbuffer or a stream engine. In such cases, the address may be a hit if itis in the same cache block as an address in the buffer or stream engine,and may be a miss otherwise. The cache block is the unit of allocationand deallocation in the cache.

The prefetch unit as described herein may be implemented in many typesof processors. An overview of one embodiment of a multi-core,multi-threaded processor is provided below with regard to FIGS. 1 and 2.However, other embodiments may be used in single core processors(multi-threaded or single-threaded), and may be used in multi-core,single-threaded processors as well.

Overview of Multithreaded Processor Architecture

A block diagram illustrating one embodiment of a multithreaded processor10 is shown in FIG. 1. In the illustrated embodiment, processor 10includes a number of processor cores 100 a-n, which are also designated“core 0” though “core n.” Various embodiments of processor 10 mayinclude varying numbers of cores 100, such as 8, 16, or any othersuitable number. Each of cores 100 is coupled to a corresponding L2cache 105 a-n, which in turn couple to L3 cache 120 via a crossbar 110.Cores 100 a-n and L2 caches 105 a-n may be generically referred to,either collectively or individually, as core(s) 100 and L2 cache(s) 105,respectively.

Via crossbar 110 and L3 cache 120, cores 100 may be coupled to a varietyof devices that may be located externally to processor 10. In theillustrated embodiment, one or more memory interface(s) 130 may beconfigured to couple to one or more banks of system memory (not shown).One or more coherent processor interface(s) 140 may be configured tocouple processor 10 to other processors (e.g., in a multiprocessorenvironment employing multiple units of processor 10). Additionally,system interconnect 125 couples cores 100 to one or more peripheralinterface(s) 150 and network interface(s) 160. As described in greaterdetail below, these interfaces may be configured to couple processor 10to various peripheral devices and networks.

Cores 100 may be configured to execute instructions and to process dataaccording to a particular instruction set architecture (ISA). In oneembodiment, cores 100 may be configured to implement a version of theSPARC® ISA, such as SPARC® V9, UltraSPARC Architecture 2005, UltraSPARCArchitecture 2007, or UltraSPARC Architecture 2009, for example.However, in other embodiments it is contemplated that any desired ISAmay be employed, such as x86 (32-bit or 64-bit versions), PowerPC® orMIPS®, for example.

In the illustrated embodiment, each of cores 100 may be configured tooperate independently of the others, such that all cores 100 may executein parallel. Additionally, as described below in conjunction with thedescription of FIG. 2, in some embodiments, each of cores 100 may beconfigured to execute multiple threads concurrently, where a giventhread may include a set of instructions that may execute independentlyof instructions from another thread. (For example, an individualsoftware process, such as an application, may consist of one or morethreads that may be scheduled for execution by an operating system.)Such a core 100 may also be referred to as a multithreaded (MT) core. Inone embodiment, each of cores 100 may be configured to concurrentlyexecute instructions from a variable number of threads, up to eightconcurrently-executing threads. In a 16-core implementation, processor10 could thus concurrently execute up to 128 threads. However, in otherembodiments it is contemplated that other numbers of cores 100 may beprovided, and that cores 100 may concurrently process different numbersof threads.

Additionally, as described in greater detail below, in some embodiments,each of cores 100 may be configured to execute certain instructions outof program order, which may also be referred to herein as out-of-orderexecution, or simply OOO. As an example of out-of-order execution, for aparticular thread, there may be instructions that are subsequent inprogram order to a given instruction yet do not depend on the giveninstruction. If execution of the given instruction is delayed for somereason (e.g., owing to a cache miss), the later instructions may executebefore the given instruction completes, which may improve overallperformance of the executing thread.

As shown in FIG. 1, in one embodiment, each core 100 may have adedicated corresponding L2 cache 105. In one embodiment, L2 cache 105may be configured as a set-associative, writeback cache that is fullyinclusive of first-level cache state (e.g., instruction and data cacheswithin core 100). To maintain coherence with first-level caches,embodiments of L2 cache 105 may implement a reverse directory thatmaintains a virtual copy of the first-level cache tags. L2 cache 105 mayimplement a coherence protocol (e.g., the MESI protocol) to maintaincoherence with other caches within processor 10. In one embodiment, L2cache 105 may enforce a Total Store Ordering (TSO) model of execution inwhich all store instructions from the same thread must complete inprogram order.

In various embodiments, L2 cache 105 may include a variety of structuresconfigured to support cache functionality and performance. For example,L2 cache 105 may include a miss buffer configured to store requests thatmiss the L2, a fill buffer configured to temporarily store datareturning from L3 cache 120, a writeback buffer configured totemporarily store dirty evicted data and snoop copyback data, and/or asnoop buffer configured to store snoop requests received from L3 cache120. In one embodiment, L2 cache 105 may implement a history-basedprefetcher that may attempt to analyze L2 miss behavior andcorrespondingly generate prefetch requests to L3 cache 120.

Crossbar 110 may be configured to manage data flow between L2 caches 105and the shared L3 cache 120. In one embodiment, crossbar 110 may includelogic (such as multiplexers or a switch fabric, for example) that allowsany L2 cache 105 to access any bank of L3 cache 120, and that converselyallows data to be returned from any L3 bank to any L2 cache 105. Thatis, crossbar 110 may be configured as an M-to-N crossbar that allows forgeneralized point-to-point communication. However, in other embodiments,other interconnection schemes may be employed between L2 caches 105 andL3 cache 120. For example, a mesh, ring, or other suitable topology maybe utilized.

Crossbar 110 may be configured to concurrently process data requestsfrom L2 caches 105 to L3 cache 120 as well as data responses from L3cache 120 to L2 caches 105. In some embodiments, crossbar 110 mayinclude logic to queue data requests and/or responses, such thatrequests and responses may not block other activity while waiting forservice. Additionally, in one embodiment crossbar 110 may be configuredto arbitrate conflicts that may occur when multiple L2 caches 105attempt to access a single bank of L3 cache 120, or vice versa.

L3 cache 120 may be configured to cache instructions and data for use bycores 100. In the illustrated embodiment, L3 cache 120 may be organizedinto eight separately addressable banks that may each be independentlyaccessed, such that in the absence of conflicts, each bank mayconcurrently return data to a respective L2 cache 105. In someembodiments, each individual bank may be implemented usingset-associative or direct-mapped techniques. For example, in oneembodiment, L3 cache 120 may be an 8 megabyte (MB) cache, where each 1MB bank is 16-way set associative with a 64-byte line size. L3 cache 120may be implemented in some embodiments as a writeback cache in whichwritten (dirty) data may not be written to system memory until acorresponding cache line is evicted. However, it is contemplated that inother embodiments, L3 cache 120 may be configured in any suitablefashion. For example, L3 cache 120 may be implemented with more or fewerbanks, or in a scheme that does not employ independently-accessiblebanks; it may employ other bank sizes or cache geometries (e.g.,different line sizes or degrees of set associativity); it may employwrite-through instead of writeback behavior; and it may or may notallocate on a write miss. Other variations of L3 cache 120 configurationare possible and contemplated.

In some embodiments, L3 cache 120 may implement queues for requestsarriving from and results to be sent to crossbar 110. Additionally, insome embodiments L3 cache 120 may implement a fill buffer configured tostore fill data arriving from memory interface 130, a writeback bufferconfigured to store dirty evicted data to be written to memory, and/or amiss buffer configured to store L3 cache accesses that cannot beprocessed as simple cache hits (e.g., L3 cache misses, cache accessesmatching older misses, accesses such as atomic operations that mayrequire multiple cache accesses, etc.). L3 cache 120 may variously beimplemented as single-ported or multiported (i.e., capable of processingmultiple concurrent read and/or write accesses). In either case, L3cache 120 may implement arbitration logic to prioritize cache accessamong various cache read and write requesters.

Not all external accesses from cores 100 necessarily proceed through L3cache 120. In the illustrated embodiment, non-cacheable unit (NCU) 122may be configured to process requests from cores 100 for non-cacheabledata, such as data from I/O devices as described below with respect toperipheral interface(s) 150 and network interface(s) 160.

Memory interface 130 may be configured to manage the transfer of databetween L3 cache 120 and system memory, for example in response to cachefill requests and data evictions. In some embodiments, multipleinstances of memory interface 130 may be implemented, with each instanceconfigured to control a respective bank of system memory. Memoryinterface 130 may be configured to interface to any suitable type ofsystem memory, such as Fully Buffered Dual Inline Memory Module(FB-DIMM), Double Data Rate or Double Data Rate 2, 3, or 4 SynchronousDynamic Random Access Memory (DDR/DDR2/DDR3/DDR4 SDRAM), or Rambus® DRAM(RDRAM®), for example. In some embodiments, memory interface 130 may beconfigured to support interfacing to multiple different types of systemmemory. In the illustrated embodiment, processor 10 may also beconfigured to receive data from sources other than system memory. Systeminterconnect 125 may be configured to provide a central interface forsuch sources to exchange data with cores 100, L2 caches 105, and/or L3cache 120. In some embodiments, system interconnect 125 may beconfigured to coordinate Direct Memory Access (DMA) transfers of data toand from system memory. For example, via memory interface 130, systeminterconnect 125 may coordinate DMA transfers between system memory anda network device attached via network interface 160, or between systemmemory and a peripheral device attached via peripheral interface 150.

Processor 10 may be configured for use in a multiprocessor environmentwith other instances of processor 10 or other compatible processors. Inthe illustrated embodiment, coherent processor interface(s) 140 may beconfigured to implement high-bandwidth, direct chip-to-chipcommunication between different processors in a manner that preservesmemory coherence among the various processors (e.g., according to acoherence protocol that governs memory transactions).

Peripheral interface 150 may be configured to coordinate data transferbetween processor 10 and one or more peripheral devices. Such peripheraldevices may include, for example and without limitation, storage devices(e.g., magnetic or optical media-based storage devices including harddrives, tape drives, CD drives, DVD drives, etc.), display devices(e.g., graphics subsystems), multimedia devices (e.g., audio processingsubsystems), or any other suitable type of peripheral device. In oneembodiment, peripheral interface 150 may implement one or more instancesof a standard peripheral interface. For example, one embodiment ofperipheral interface 150 may implement the Peripheral ComponentInterface Express (PCI Express™ or PCIe) standard according togeneration 1.x, 2.0, 3.0, or another suitable variant of that standard,with any suitable number of I/O lanes. However, it is contemplated thatany suitable interface standard or combination of standards may beemployed. For example, in some embodiments peripheral interface 150 maybe configured to implement a version of Universal Serial Bus (USB)protocol or IEEE 1394 (Firewire®) protocol in addition to or instead ofPCI Express™.

Network interface 160 may be configured to coordinate data transferbetween processor 10 and one or more network devices (e.g., networkedcomputer systems or peripherals) coupled to processor 10 via a network.In one embodiment, network interface 160 may be configured to performthe data processing necessary to implement an Ethernet (IEEE 802.3)networking standard such as Gigabit Ethernet or 10-Gigabit Ethernet, forexample. However, it is contemplated that any suitable networkingstandard may be implemented, including forthcoming standards such as40-Gigabit Ethernet and 100-Gigabit Ethernet. In some embodiments,network interface 160 may be configured to implement other types ofnetworking protocols, such as Fibre Channel, Fibre Channel over Ethernet(FCoE), Data Center Ethernet, Infiniband, and/or other suitablenetworking protocols. In some embodiments, network interface 160 may beconfigured to implement multiple discrete network interface ports.

Overview of Dynamic Multithreading Processor Core

As mentioned above, in one embodiment each of cores 100 may beconfigured for multithreaded, out-of-order execution. More specifically,in one embodiment, each of cores 100 may be configured to performdynamic multithreading. Generally speaking, under dynamicmultithreading, the execution resources of cores 100 may be configuredto efficiently process varying types of computational workloads thatexhibit different performance characteristics and resource requirements.Such workloads may vary across a continuum that emphasizes differentcombinations of individual-thread and multiple-thread performance.

At one end of the continuum, a computational workload may include anumber of independent tasks, where completing the aggregate set of taskswithin certain performance criteria (e.g., an overall number of tasksper second) is a more significant factor in system performance than therate at which any particular task is completed. For example, in certaintypes of server or transaction processing environments, there may be ahigh volume of individual client or customer requests (such as web pagerequests or file system accesses). In this context, individual requestsmay not be particularly sensitive to processor performance. For example,requests may be I/O-bound rather than processor-bound—completion of anindividual request may require I/O accesses (e.g., to relatively slowmemory, network, or storage devices) that dominate the overall timerequired to complete the request, relative to the processor effortinvolved. Thus, a processor that is capable of concurrently processingmany such tasks (e.g., as independently executing threads) may exhibitbetter performance on such a workload than a processor that emphasizesthe performance of only one or a small number of concurrent tasks.

At the other end of the continuum, a computational workload may includeindividual tasks whose performance is highly processor-sensitive. Forexample, a task that involves significant mathematical analysis and/ortransformation (e.g., cryptography, graphics processing, scientificcomputing) may be more processor-bound than I/O-bound. Such tasks maybenefit from processors that emphasize single-task performance, forexample through speculative execution and exploitation ofinstruction-level parallelism.

Dynamic multithreading represents an attempt to allocate processorresources in a manner that flexibly adapts to workloads that vary alongthe continuum described above. In one embodiment, cores 100 may beconfigured to implement fine-grained multithreading, in which each coremay select instructions to execute from among a pool of instructionscorresponding to multiple threads, such that instructions from differentthreads may be scheduled to execute adjacently. For example, in apipelined embodiment of core 100 employing fine-grained multithreading,instructions from different threads may occupy adjacent pipeline stages,such that instructions from several threads may be in various stages ofexecution during a given core processing cycle. Through the use offine-grained multithreading, cores 100 may be configured to efficientlyprocess workloads that depend more on concurrent thread processing thanindividual thread performance.

In one embodiment, cores 100 may also be configured to implementout-of-order processing, speculative execution, register renaming and/orother features that improve the performance of processor-dependentworkloads. Moreover, cores 100 may be configured to dynamically allocatea variety of hardware resources among the threads that are activelyexecuting at a given time, such that if fewer threads are executing,each individual thread may be able to take advantage of a greater shareof the available hardware resources. This may result in increasedindividual thread performance when fewer threads are executing, whileretaining the flexibility to support workloads that exhibit a greaternumber of threads that are less processor-dependent in theirperformance. In various embodiments, the resources of a given core 100that may be dynamically allocated among a varying number of threads mayinclude branch resources (e.g., branch predictor structures), load/storeresources (e.g., load/store buffers and queues), instruction completionresources (e.g., reorder buffer structures and commit logic),instruction issue resources (e.g., instruction selection and schedulingstructures), register rename resources (e.g., register mapping tables),and/or memory management unit resources (e.g., translation lookasidebuffers, page walk resources).

One embodiment of core 100 that is configured to perform dynamicmultithreading is illustrated in FIG. 2. In the illustrated embodiment,core 100 includes an instruction fetch unit (IFU) 200 that includes aninstruction cache 205. IFU 200 is coupled to a memory management unit(MMU) 270, L2 interface 265, and trap logic unit (TLU) 275. IFU 200 isadditionally coupled to an instruction processing pipeline that beginswith a select unit 210 and proceeds in turn through a decode unit 215, arename unit 220, a pick unit 225, and an issue unit 230. Issue unit 230is coupled to issue instructions to any of a number of instructionexecution resources: an execution unit 0 (EXU0) 235, an execution unit 1(EXU1) 240, a load store unit (LSU) 245 that includes a data cache 250,and/or a floating point/graphics unit (FGU) 255. These instructionexecution resources are coupled to a working register file 260.Additionally, LSU 245 is coupled to L2 interface 265 and MMU 270.

In the following discussion, exemplary embodiments of each of thestructures of the illustrated embodiment of core 100 are described.However, it is noted that the illustrated partitioning of resources ismerely one example of how core 100 may be implemented. Alternativeconfigurations and variations are possible and contemplated.

Instruction fetch unit 200 may be configured to provide instructions tothe rest of core 100 for execution. In one embodiment, IFU 200 may beconfigured to select a thread to be fetched, fetch instructions frominstruction cache 205 for the selected thread and buffer them fordownstream processing, request data from L2 cache 105 in response toinstruction cache misses, and predict the direction and target ofcontrol transfer instructions (e.g., branches). In some embodiments, IFU200 may include a number of data structures in addition to instructioncache 205, such as an instruction translation lookaside buffer (ITLB),instruction buffers, and/or structures configured to store state that isrelevant to thread selection and processing.

In one embodiment, during each execution cycle of core 100, IFU 200 maybe configured to select one thread that will enter the IFU processingpipeline. Thread selection may take into account a variety of factorsand conditions, some thread-specific and others IFU-specific. Forexample, certain instruction cache activities (e.g., cache fill), ITLBactivities, or diagnostic activities may inhibit thread selection ifthese activities are occurring during a given execution cycle.Additionally, individual threads may be in specific states of readinessthat affect their eligibility for selection. For example, a thread forwhich there is an outstanding instruction cache miss may not be eligiblefor selection until the miss is resolved. In some embodiments, thosethreads that are eligible to participate in thread selection may bedivided into groups by priority, for example depending on the state ofthe thread or of the ability of the IFU pipeline to process the thread.In such embodiments, multiple levels of arbitration may be employed toperform thread selection: selection occurs first by group priority, andthen within the selected group according to a suitable arbitrationalgorithm (e.g., a least-recently-fetched algorithm). However, it isnoted that any suitable scheme for thread selection may be employed,including arbitration schemes that are more complex or simpler thanthose mentioned here.

Once a thread has been selected for fetching by IFU 200, instructionsmay actually be fetched for the selected thread. To perform the fetch,in one embodiment, IFU 200 may be configured to generate a fetch addressto be supplied to instruction cache 205. In various embodiments, thefetch address may be generated as a function of a program counterassociated with the selected thread, a predicted branch target address,or an address supplied in some other manner (e.g., through a test ordiagnostic mode). The generated fetch address may then be applied toinstruction cache 205 to determine whether there is a cache hit.

In some embodiments, accessing instruction cache 205 may includeperforming fetch address translation (e.g., in the case of a physicallyindexed and/or tagged cache), accessing a cache tag array, and comparinga retrieved cache tag to a requested tag to determine cache hit status.If there is a cache hit, IFU 200 may store the retrieved instructionswithin buffers for use by later stages of the instruction pipeline. Ifthere is a cache miss, IFU 200 may coordinate retrieval of the missingcache data from L2 cache 105. In some embodiments, IFU 200 may also beconfigured to prefetch instructions into instruction cache 205 beforethe instructions are actually required to be fetched. For example, inthe case of a cache miss, IFU 200 may be configured to retrieve themissing data for the requested fetch address as well as addresses thatsequentially follow the requested fetch address, on the assumption thatthe following addresses are likely to be fetched in the near future.

In many ISAs, instruction execution proceeds sequentially according toinstruction addresses (e.g., as reflected by one or more programcounters). However, control transfer instructions (CTIs) such asbranches, call/return instructions, or other types of instructions maycause the transfer of execution from a current fetch address to anonsequential address. As mentioned above, IFU 200 may be configured topredict the direction and target of CTIs (or, in some embodiments, asubset of the CTIs that are defined for an ISA) in order to reduce thedelays incurred by waiting until the effect of a CTI is known withcertainty. In one embodiment, IFU 200 may be configured to implement aperceptron-based dynamic branch predictor, although any suitable type ofbranch predictor may be employed.

To implement branch prediction, IFU 200 may implement a variety ofcontrol and data structures in various embodiments, such as historyregisters that track prior branch history, weight tables that reflectrelative weights or strengths of predictions, and/or target datastructures that store fetch addresses that are predicted to be targetsof a CTI. Also, in some embodiments, IFU 200 may further be configuredto partially decode (or predecode) fetched instructions in order tofacilitate branch prediction. A predicted fetch address for a giventhread may be used as the fetch address when the given thread isselected for fetching by IFU 200. The outcome of the prediction may bevalidated when the CTI is actually executed (e.g., if the CTI is aconditional instruction, or if the CTI itself is in the path of anotherpredicted CTI). If the prediction was incorrect, instructions along thepredicted path that were fetched and issued may be cancelled.

Through the operations discussed above, IFU 200 may be configured tofetch and maintain a buffered pool of instructions from one or multiplethreads, to be fed into the remainder of the instruction pipeline forexecution. Generally speaking, select unit 210 may be configured toselect and schedule threads for execution. In one embodiment, during anygiven execution cycle of core 100, select unit 210 may be configured toselect up to one ready thread out of the maximum number of threadsconcurrently supported by core 100 (e.g., 8 threads), and may select upto two instructions from the selected thread for decoding by decode unit215, although in other embodiments, a differing number of threads andinstructions may be selected. In various embodiments, differentconditions may affect whether a thread is ready for selection by selectunit 210, such as branch mispredictions, unavailable instructions, orother conditions. To ensure fairness in thread selection, someembodiments of select unit 210 may employ arbitration among readythreads (e.g. a least-recently-used algorithm).

The particular instructions that are selected for decode by select unit210 may be subject to the decode restrictions of decode unit 215; thus,in any given cycle, fewer than the maximum possible number ofinstructions may be selected. Additionally, in some embodiments, selectunit 210 may be configured to allocate certain execution resources ofcore 100 to the selected instructions, so that the allocated resourceswill not be used for the benefit of another instruction until they arereleased. For example, select unit 210 may allocate resource tags forentries of a reorder buffer, load/store buffers, or other downstreamresources that may be utilized during instruction execution.

Generally, decode unit 215 may be configured to prepare the instructionsselected by select unit 210 for further processing. Decode unit 215 maybe configured to identify the particular nature of an instruction (e.g.,as specified by its opcode) and to determine the source and sink (i.e.,destination) registers encoded in an instruction, if any. In someembodiments, decode unit 215 may be configured to detect certaindependencies among instructions, to remap architectural registers to aflat register space, and/or to convert certain complex instructions totwo or more simpler instructions for execution. Additionally, in someembodiments, decode unit 215 may be configured to assign instructions toslots for subsequent scheduling. In one embodiment, two slots 0-1 may bedefined, where slot 0 includes instructions executable in load/storeunit 245 or execution units 235-240, and where slot 1 includesinstructions executable in execution units 235-240, floatingpoint/graphics unit 255, and any branch instructions. However, in otherembodiments, other numbers of slots and types of slot assignments may beemployed, or slots may be omitted entirely.

Register renaming may facilitate the elimination of certain dependenciesbetween instructions (e.g., write-after-read or “false” dependencies),which may in turn prevent unnecessary serialization of instructionexecution. In one embodiment, rename unit 220 may be configured torename the logical (i.e., architected) destination registers specifiedby instructions by mapping them to a physical register space, resolvingfalse dependencies in the process. In some embodiments, rename unit 220may maintain mapping tables that reflect the relationship betweenlogical registers and the physical registers to which they are mapped.

Once decoded and renamed, instructions may be ready to be scheduled forexecution. In the illustrated embodiment, pick unit 225 may beconfigured to pick instructions that are ready for execution and sendthe picked instructions to issue unit 230. In one embodiment, pick unit225 may be configured to maintain a pick queue that stores a number ofdecoded and renamed instructions as well as information about therelative age and status of the stored instructions. During eachexecution cycle, this embodiment of pick unit 225 may pick up to oneinstruction per slot. For example, taking instruction dependency and ageinformation into account, for a given slot, pick unit 225 may beconfigured to pick the oldest instruction for the given slot that isready to execute.

In some embodiments, pick unit 225 may be configured to supportload/store speculation by retaining speculative load/store instructions(and, in some instances, their dependent instructions) after they havebeen picked. This may facilitate replaying of instructions in the eventof load/store misspeculation. Additionally, in some embodiments, pickunit 225 may be configured to deliberately insert “holes” into thepipeline through the use of stalls, e.g., in order to manage downstreampipeline hazards such as synchronization of certain load/store orlong-latency FGU instructions.

Issue unit 230 may be configured to provide instruction sources and datato the various execution units for picked instructions. In oneembodiment, issue unit 230 may be configured to read source operandsfrom the appropriate source, which may vary depending upon the state ofthe pipeline. For example, if a source operand depends on a priorinstruction that is still in the execution pipeline, the operand may bebypassed directly from the appropriate execution unit result bus.Results may also be sourced from register files representingarchitectural (i.e., user-visible) as well as non-architectural state.In the illustrated embodiment, core 100 includes a working register file260 that may be configured to store instruction results (e.g., integerresults, floating point results, and/or condition code results) thathave not yet been committed to architectural state, and which may serveas the source for certain operands. The various execution units may alsomaintain architectural integer, floating-point, and condition code statefrom which operands may be sourced.

Instructions issued from issue unit 230 may proceed to one or more ofthe illustrated execution units for execution. In one embodiment, eachof EXU0 235 and EXU1 240 may be similarly or identically configured toexecute certain integer-type instructions defined in the implementedISA, such as arithmetic, logical, and shift instructions. In theillustrated embodiment, EXU0 235 may be configured to execute integerinstructions issued from slot 0, and may also perform addresscalculation and for load/store instructions executed by LSU 245. EXU1240 may be configured to execute integer instructions issued from slot1, as well as branch instructions. In one embodiment, FGU instructionsand multicycle integer instructions may be processed as slot 1instructions that pass through the EXU1 240 pipeline, although some ofthese instructions may actually execute in other functional units.

In some embodiments, architectural and non-architectural register filesmay be physically implemented within or near execution units 235-240. Itis contemplated that in some embodiments, core 100 may include more orfewer than two integer execution units, and the execution units may ormay not be symmetric in functionality. Also, in some embodimentsexecution units 235-240 may not be bound to specific issue slots, or maybe differently bound than just described.

Load store unit 245 may be configured to process data memory references,such as integer and floating-point load and store instructions and othertypes of memory reference instructions. LSU 245 may include a data cache250 as well as logic configured to detect data cache misses and toresponsively request data from L2 cache 105. In one embodiment, datacache 250 may be configured as a set-associative, write-through cache inwhich all stores are written to L2 cache 105 regardless of whether theyhit in data cache 250. As noted above, the actual computation ofaddresses for load/store instructions may take place within one of theinteger execution units, though in other embodiments, LSU 245 mayimplement dedicated address generation logic. In some embodiments, LSU245 may implement an adaptive, history-dependent hardware prefetcherconfigured to predict and prefetch data that is likely to be used in thefuture, in order to increase the likelihood that such data will beresident in data cache 250 when it is needed.

In various embodiments, LSU 245 may implement a variety of structuresconfigured to facilitate memory operations. For example, LSU 245 mayimplement a data TLB to cache virtual data address translations, as wellas load and store buffers configured to store issued butnot-yet-committed load and store instructions for the purposes ofcoherency snooping and dependency checking. LSU 245 may include a missbuffer configured to store outstanding loads and stores that cannot yetcomplete, for example due to cache misses. In one embodiment, LSU 245may implement a store queue configured to store address and datainformation for stores that have committed, in order to facilitate loaddependency checking. LSU 245 may also include hardware configured tosupport atomic load-store instructions, memory-related exceptiondetection, and read and write access to special-purpose registers (e.g.,control registers).

Floating point/graphics unit 255 may be configured to execute andprovide results for certain floating-point and graphics-orientedinstructions defined in the implemented ISA. For example, in oneembodiment FGU 255 may implement single- and double-precisionfloating-point arithmetic instructions compliant with the IEEE 754-1985floating-point standard, such as add, subtract, multiply, divide, andcertain transcendental functions. Also, in one embodiment FGU 255 mayimplement partitioned-arithmetic and graphics-oriented instructionsdefined by a version of the SPARC® Visual Instruction Set (VIS™)architecture, such as VIS™ 2.0 or VIS™ 3.0. In some embodiments, FGU 255may implement fused and unfused floating-point multiply-addinstructions. Additionally, in one embodiment FGU 255 may implementcertain integer instructions such as integer multiply, divide, andpopulation count instructions. Depending on the implementation of FGU255, some instructions (e.g., some transcendental or extended-precisioninstructions) or instruction operand or result scenarios (e.g., certaindenormal operands or expected results) may be trapped and handled oremulated by software.

In one embodiment, FGU 255 may implement separate execution pipelinesfor floating point add/multiply, divide/square root, and graphicsoperations, while in other embodiments the instructions implemented byFGU 255 may be differently partitioned. In various embodiments,instructions implemented by FGU 255 may be fully pipelined (i.e., FGU255 may be capable of starting one new instruction per execution cycle),partially pipelined, or may block issue until complete, depending on theinstruction type. For example, in one embodiment floating-point add andmultiply operations may be fully pipelined, while floating-point divideoperations may block other divide/square root operations untilcompleted.

Embodiments of FGU 255 may also be configured to implement hardwarecryptographic support. For example, FGU 255 may include logic configuredto support encryption/decryption algorithms such as Advanced EncryptionStandard (AES), Data Encryption Standard/Triple Data Encryption Standard(DES/3DES), the Kasumi block cipher algorithm, and/or the Camellia blockcipher algorithm. FGU 255 may also include logic to implement hash orchecksum algorithms such as Secure Hash Algorithm (SHA-1, SHA-256,SHA-384, SHA-512), or Message Digest 5 (MD5). FGU 255 may also beconfigured to implement modular arithmetic such as modularmultiplication, reduction and exponentiation, as well as various typesof Galois field operations. In one embodiment, FGU 255 may be configuredto utilize the floating-point multiplier array for modularmultiplication. In various embodiments, FGU 255 may implement several ofthe aforementioned algorithms as well as other algorithms notspecifically described.

The various cryptographic and modular arithmetic operations provided byFGU 255 may be invoked in different ways for different embodiments. Inone embodiment, these features may be implemented via a discretecoprocessor that may be indirectly programmed by software, for exampleby using a control word queue defined through the use of specialregisters or memory-mapped registers. In another embodiment, the ISA maybe augmented with specific instructions that may allow software todirectly perform these operations.

As previously described, instruction and data memory accesses mayinvolve translating virtual addresses to physical addresses. In oneembodiment, such translation may occur on a page level of granularity,where a certain number of address bits comprise an offset into a givenpage of addresses, and the remaining address bits comprise a pagenumber. For example, in an embodiment employing 4 MB pages, a 64-bitvirtual address and a 40-bit physical address, 22 address bits(corresponding to 4 MB of address space, and typically the leastsignificant address bits) may constitute the page offset. The remaining42 bits of the virtual address may correspond to the virtual page numberof that address, and the remaining 18 bits of the physical address maycorrespond to the physical page number of that address. In such anembodiment, virtual to physical address translation may occur by mappinga virtual page number to a particular physical page number, leaving thepage offset unmodified.

Such translation mappings may be stored in an ITLB or a DTLB for rapidtranslation of virtual addresses during lookup of instruction cache 205or data cache 250. In the event no translation for a given virtual pagenumber is found in the appropriate TLB, memory management unit 270 maybe configured to provide a translation. In one embodiment, MMU 270 maybe configured to manage one or more translation tables stored in systemmemory and to traverse such tables (which in some embodiments may behierarchically organized) in response to a request for an addresstranslation, such as from an ITLB or DTLB miss. (Such a traversal mayalso be referred to as a page table walk or a hardware table walk.) Insome embodiments, if MMU 270 is unable to derive a valid addresstranslation, for example if one of the memory pages including anecessary page table is not resident in physical memory (i.e., a pagemiss), MMU 270 may be configured to generate a trap to allow a memorymanagement software routine to handle the translation. It iscontemplated that in various embodiments, any desirable page size may beemployed. Further, in some embodiments multiple page sizes may beconcurrently supported.

As noted above, several functional units in the illustrated embodimentof core 100 may be configured to generate off-core memory requests. Forexample, IFU 200 and LSU 245 each may generate access requests to L2cache 105 in response to their respective cache misses. Additionally,MMU 270 may be configured to generate memory requests, for example whileexecuting a page table walk. In the illustrated embodiment, L2 interface265 may be configured to provide a centralized interface to the L2 cache105 associated with a particular core 100, on behalf of the variousfunctional units that may generate L2 accesses. In one embodiment, L2interface 265 may be configured to maintain queues of pending L2requests and to arbitrate among pending requests to determine whichrequest or requests may be conveyed to L2 cache 105 during a givenexecution cycle. For example, L2 interface 265 may implement aleast-recently-used or other algorithm to arbitrate among L2 requesters.In one embodiment, L2 interface 265 may also be configured to receivedata returned from L2 cache 105, and to direct such data to theappropriate functional unit (e.g., to data cache 250 for a data cachefill due to miss).

During the course of operation of some embodiments of core 100,exceptional events may occur. For example, an instruction from a giventhread that is selected for execution by select unit 210 may not be avalid instruction for the ISA implemented by core 100 (e.g., theinstruction may have an illegal opcode), a floating-point instructionmay produce a result that requires further processing in software, MMU270 may not be able to complete a page table walk due to a page miss, ahardware error (such as uncorrectable data corruption in a cache orregister file) may be detected, or any of numerous other possiblearchitecturally-defined or implementation-specific exceptional eventsmay occur. In one embodiment, trap logic unit 275 may be configured tomanage the handling of such events. For example, TLU 275 may beconfigured to receive notification of an exceptional event occurringduring execution of a particular thread, and to cause execution controlof that thread to vector to a supervisor-mode software handler (i.e., atrap handler) corresponding to the detected event. Such handlers mayinclude, for example, an illegal opcode trap handler configured toreturn an error status indication to an application associated with thetrapping thread and possibly terminate the application, a floating-pointtrap handler configured to fix up an inexact result, etc.

In one embodiment, TLU 275 may be configured to flush all instructionsfrom the trapping thread from any stage of processing within core 100,without disrupting the execution of other, non-trapping threads. In someembodiments, when a specific instruction from a given thread causes atrap (as opposed to a trap-causing condition independent of instructionexecution, such as a hardware interrupt request), TLU 275 may implementsuch traps as precise traps. That is, TLU 275 may ensure that allinstructions from the given thread that occur before the trappinginstruction (in program order) complete and update architectural state,while no instructions from the given thread that occur after thetrapping instruction (in program) order complete or update architecturalstate.

Additionally, in the absence of exceptions or trap requests, TLU 275 maybe configured to initiate and monitor the commitment of working resultsto architectural state. For example, TLU 275 may include a reorderbuffer (ROB) that coordinates transfer of speculative results intoarchitectural state. TLU 275 may also be configured to coordinate threadflushing that results from branch misprediction. For instructions thatare not flushed or otherwise cancelled due to mispredictions orexceptions, instruction processing may end when instruction results havebeen committed.

In various embodiments, any of the units illustrated in FIG. 2 may beimplemented as one or more pipeline stages, to form an instructionexecution pipeline that begins when thread fetching occurs in IFU 200and ends with result commitment by TLU 275. Depending on the manner inwhich the functionality of the various units of FIG. 2 is partitionedand implemented, different units may require different numbers of cyclesto complete their portion of instruction processing. In some instances,certain units (e.g., FGU 255) may require a variable number of cycles tocomplete certain types of operations.

Through the use of dynamic multithreading, in some instances, it ispossible for each stage of the instruction pipeline of core 100 to holdan instruction from a different thread in a different stage ofexecution, in contrast to conventional processor implementations thattypically require a pipeline flush when switching between threads orprocesses. In some embodiments, flushes and stalls due to resourceconflicts or other scheduling hazards may cause some pipeline stages tohave no instruction during a given cycle. However, in the fine-grainedmultithreaded processor implementation employed by the illustratedembodiment of core 100, such flushes and stalls may be directed to asingle thread in the pipeline, leaving other threads undisturbed.Additionally, even if one thread being processed by core 100 stalls fora significant length of time (for example, due to an L2 cache miss),instructions from another thread may be readily selected for issue, thusincreasing overall thread processing throughput.

As described previously, however, the various resources of core 100 thatsupport fine-grained multithreaded execution may also be dynamicallyreallocated to improve the performance of workloads having fewer numbersof threads. Under these circumstances, some threads may be allocated alarger share of execution resources while other threads are allocatedcorrespondingly fewer resources. Even when fewer threads are sharingcomparatively larger shares of execution resources, however, core 100may still exhibit the flexible, thread-specific flush and stall behaviordescribed above.

Prefetch Unit

Turning now to FIG. 3, a block diagram illustrating one embodiment ofthe data cache 250 and a prefetch unit 300 (which may be part of the LSU245, in one embodiment) is shown. The data cache 250 and the prefetchunit 300 (and more particularly the stream engines 306A-306N and the PLTcontrol circuit 302) are coupled to receive a data address correspondingto a memory operation, and the data cache 250 is configured to transmita hit signal to the prefetch unit 300 (and more particularly to the PLTcontrol circuit 302 in the prefetch unit 300). The prefetch unit 300(and more particularly the prefetch learning table memory 304 and thePLT control circuit 302) is coupled to receive a PC and a thread ID(TID). The prefetch unit 300 includes the PLT control circuit 302, thePLT memory 304, the plurality of stream engines 306A-306N, a prefetchcontrol circuit 308, and a multiplexor (mux) 310. The PLT controlcircuit 302 is coupled to the PLT memory 304 and to receive additionalhit signals from a miss request buffer (MRB) and the stream engines306A-306N. The PLT memory 304 is coupled to the stream engines 306A-306Nand to the prefetch control circuit 308, which is coupled to receivepipe status for the load/store pipe (L/S pipe status in FIG. 3). Theprefetch control unit is coupled to the stream engines 306A-306N and toprovide selection control for the mux 310. The inputs to the mux 310 arethe prefetch outputs from the stream engines 306A-306N, and the outputof the mux 310 is an issued prefetch. The stream engine 306A is shown inmore detail in FIG. 3 to include an SE control circuit 312 and a set ofregisters 314.

As mentioned previously, the prefetch unit 300 may be configured tomonitor the cache misses from the data cache 250 to identify prefetchstreams. Accordingly, the data address that is supplied to the datacache 250 may be provided to the prefetch unit 300, along with the hitsignal from the data cache 250. If the data address is a miss in thedata cache 250, the address may be a candidate from which to attempt todetect a prefetch stream. In one embodiment, the data address may bequalified by whether or not the data address is for a load operation(load signal to the PLT control circuit 302 and the stream engines306A-306N in FIG. 3). The load signal may be asserted for a loadoperation, and may be deasserted for a store operation, a softwareprefetch operation, and a prefetch issued by the prefetch unit 300 thatis being filtered through the data cache 250.

In the illustrated embodiment, a data address of a load operation thatis a cache miss is also compared to the miss request buffer in theprocessor core (not shown in FIG. 3). The miss request buffer stores oneor more addresses of cache misses (as well as other operations such asnon-cacheable operations) that are to be transmitted to the next lowerlevel in the memory hierarchy or elsewhere outside the core 100. Thecache misses may result in cache fills to retrieve the missing cacheblocks and write the missing blocks to the data cache 250. Accordingly,if another cache miss to the same cache block is detected, the prefetchunit 300 has learned of that cache miss and need not monitor the addressfor prefetch purposes. Additionally, the data address is compared to theaddresses in the stream engines 306A-306N. If the data address isincluded in an already-allocated prefetch stream, then it may not needto be monitored for additional prefetch purposes.

The PC and TID of the load operation are provided to the PLT memory 304to select one or more entries. The PLT memory 304 may have anyconfiguration, including configurations similar to a cache. For example,the PLT memory 304 may have a direct mapped configuration in which oneentry is selected for a given input address; the PLT memory 304 may havea set associative configuration in which a set of two or more entries isselected for the given input address; or the PLT memory 304 may have afully associative configuration in which the input address is comparedto a corresponding field in each entry to detect a hit. Generally, atleast a portion of the input address (the index) may be used to selectone or more entries. A remaining portion of the input address may becompared to a tag value in the entry or entries to detect a hit, if lessthan the full input address is used as the index. The data address thatmay be written to an entry may be generated separately, and is not usedto index the PLT memory 304 in this embodiment. That is, the inputaddress is not the data address in this embodiment.

As mentioned previously, the PC (and optionally the TID) may be hashedto generate the input address to the PLT memory 304. For example,selected bits of the PC and the TID may be exclusive ORed to producerespective bits of the hash value.

The PLT memory 304 may be configured to output the data from theselected entry/entries to the PLT control circuit 302 for processing andpossible update. If the data address is a data cache miss (and missesthe stream engines 306A-306N and the MRB), the PLT control circuit 302may be configured to generate an update for the PLT memory 304. If thetag match in the PLT memory 304 is a hit, the cache miss is another missfor (possibly) the same load operation, and the data may be updated tofurther attempt to discover and/or confirm a prefetch pattern. In casesin which a hashed value of the PC (and optionally TID) is used as theaddress input, there may be some aliasing but the prefetch unit 300 maybe configured to operate as if the same load is hitting in the entryeach time. If the tag match in the PLT memory 304 is a miss, the PLTcontrol circuit 302 may be configured to overwrite the entry with newdata related to the current miss. If more than one entry is selected(e.g. in a set associative embodiment), any replacement scheme may beused to select one of the entries for replacement.

The update may confirm the prefetch stream. A confirmed prefetch streammay be a prefetch stream for which demand accesses have been detectedoften enough to indicate that the pattern has been detected. Forexample, in one embodiment, at least 3 demand accesses in the stream maybe detected. The first access initializes the entry, the second accesspermits the stride to be calculated, and the third access permits thestride to be compared to determine that it matches the previouslycomputed stride.

If the update confirms a prefetch stream, the prefetch stream may beallocated to one of the stream engines 306A-306N. In one embodiment, theprefetch control circuit 308 may be configured to select which of thestream engines 306A-306N is allocated. In another embodiment, the PLTcontrol circuit 302 may be configured to select the stream engine306A-306N. Any mechanisms for selecting a stream engine may be used. Forexample, one or more of the stream engines 306A-306N may be idle (e.g.because the outstanding prefetches have reached the limit). The prefetchstream may be allocated to one of the idle stream engines. The idlestream engine may be randomly selected, or data indicating which streamengine has been idle for the longest period of time may be captured, sothat the longest-idle stream engine may be selected. Alternatively, datamay track which stream engines have most recently issue prefetches (e.g.using a least recently used scheme or a pseudo-least recently usedscheme). In one embodiment, a used bit (U bit) scheme may be used inwhich each stream engine is marked as used when it issues a prefetch,and a stream engine that is not marked as used may be selected. When allthe stream engines are marked as used, all stream engines marked asunused and the process of marking them used begins again.

In response to an allocated prefetch stream, the PLT control circuit 302may be configured to invalidate the corresponding entry in the PLTmemory 304. Since the prefetch stream has been confirmed and allocatedto a stream engine, additional learning by the PLT may becounterproductive (e.g. the prefetch stream may be confirmed again andallocated to another stream engine). As an alternative to invalidating,the entry may record an indication that the stream has been confirmedand allocated to a stream engine.

The stream engine 306A-306N which is allocated the prefetch stream maybe configured to begin issuing prefetches. Generally, a stream enginemay comprise any circuitry that may be provided with data identifying aconfirmed prefetch scheme that may be configured to issue the prefetches(up to a limit with respect to the most recently consumed prefetch) andthat may be configured to monitor consumption of the prefetches and tocontrol the number of outstanding prefetches. In one embodiment, thestream engines 306A-306N may be similar to the stream engine 306A shownin FIG. 3 in more detail.

At a given point, one or more of the stream engines 306A-306N may beconfigured to determine that a prefetch is ready to be issued. Thestream engines 306A-306N with ready prefetches may be configured tosignal the prefetch control unit 308, and may be configured to providethe prefetches to the inputs of the mux 310. The prefetch control unit308 may be configured to select one of the ready prefetches, and may beconfigured to control the mux 310 to select the prefetch as an issuedprefetch. The prefetch control unit 308 may also signal the streamengine 306A-306N when a prefetch from that engine has been issued, sothat the stream engine may update to the next prefetch.

In one embodiment, the issued prefetches may be inserted into the L/Spipe at a point prior to the access to the data cache 250. Theprefetches may thus be filtered through the data cache 250, appearing asthe data address input (with the load signal to the prefetch unit 300deasserted). If a prefetch hits in the data cache 250, it need notpropagate to lower levels of the memory hierarchy (e.g. the L2 cache 105a-105 n, the L3 cache 120, etc.). In other embodiments, the issuedprefetches may not be passed through the data cache 250.

For embodiments that filter the prefetches through the data cache 250,the prefetch control unit 308 may be configured to monitor the L/S pipestatus, searching for idle cycles in the pipe into which prefetches maybe inserted. Since demand accesses are less speculative than prefetches,a demand access may generally be permitted to proceed and the prefetchesmay be stalled. In other embodiments, additional factors may be includedin the L/S pipe status. For example, if the MRB is becoming full (or hasreached a high watermark for prefetches), then additional prefetches maybe stalled until previous activity has been handled. If the MRB isbecoming full with demand accesses, inhibiting prefetch may be betterfor overall performance than attempting prefetch. Additionally, having ahigh watermark for prefetches may prevent prefetches from occupying toomuch bandwidth, to keep the prefetches from interfering with demandaccesses.

As shown in the stream engine 306A, the registers 314 may store a pairof pointers (P Ptr and C Ptr), a stride, and limit data that indicates acurrent limit to the number of outstanding prefetches (P Limit and PCnt). The prefetch pointer (P Ptr) may be the next address that is to beprefetched in the prefetch stream assigned to the stream engine 306A.The consumption pointer (C Ptr) is the next address in the prefetchstream that is expected to be consumed by a demand access. The stride isthe stride amount detected for the prefetch stream, and may be apositive or negative value, in some embodiments. Initially, when aprefetch stream is allocated to the stream engine 306A, both the P Ptrand the C Ptr may be set to the miss address that confirmed the prefetchstream plus the stride, and the stride is set to the confirmed stridefor the prefetch stream. The P Cnt may be initialized to zero, and the PLimit may be initialized to an initial limit for the outstandingprefetches.

The P Cnt may be a counter that tracks a number of outstandingprefetches in the prefetch stream. The P Limit may be the limit to thenumber of outstanding prefetches, but the limit may also be increased asthe number of prefetches that are consumed by demand accesses (thusindicating that the prefetches are correct) increases. Additionaldetails regarding one embodiment of the update of the limit data isprovided below with regard to FIGS. 7, 8, and 9.

The SE control circuit 312 is coupled to the registers 314, and isconfigured to generate prefetches and to update the registers 314 as theprefetches are issued. Additional details for one embodiment arediscussed below.

It is noted that, while registers 314 are described as separateregisters above, data may be combined and stored into a single registeror fewer registers than shown in FIG. 3. Generally, the registers 314may be implemented by any clocked storage devices (latches, flops,etc.).

The stride that is calculated for the prefetch stream may be determinedat any granularity. For example, the stride may be calculated at thecache block granularity. In such cases, the stride may be concatenatedwith least significant zeros to cover the cache offset portion of theaddress. For example, a 64 byte cache block would result inconcatenating the stride with 6 zeros.

Other embodiments may determine the stride at a finer granularity thanthe cache block. Any granularity may be used, down to the byte level ofgranularity, in various embodiments. Finer granularity strides maypermit stride detection for prefetch streams that are not an evenmultiple of a cache block size, and may permit earlier detection ofprefetch streams with small strides.

For example, if a stride at the granularity of a cache block is used,but the actual stride of the prefetch stream is a not an even multipleof the cache block size, the stride would appear to change periodicallyduring the prefetch stream. Consider, for example, an actual stride of0x28 (that is, hexadecimal 28) and a cache block size of 32 bytes(0x20). If the initial address in the prefetch stream is zero, the firstthree demand accesses would 0x0, 0x28, and 0x50. At a cache blockgranularity, the stride would be calculated as 0x20 (or one cacheblock). However, the next address in the stream is 0x78 (which is twocache blocks from the cache block that includes 0x50). Accordingly, theprefetch stream would prefetch the intervening cache block, which wouldnot be consumed. The prefetch stream would become idle when the limit isreached, and thus the prefetch would not be as useful as it otherwisecould be. Additionally, for small strides, multiple accesses within thesame cache block may occur. By using a finer granularity stride, thestream may be detected more quickly since the stride is more accuratelycalculated.

The data address may be a virtual address, or may be a physical addressto which the virtual address translates (e.g. through a translationlookaside buffer (TLB) or other translation caching structure). If thedata address is virtual, the prefetch unit 300 may insert prefetchesinto the L/S pipe at a point prior to initiating translation of the dataaddress. The translation may be provided prior to data cache access (inwhich case the physical address may be received as illustrated in FIG.3), or may be performed in parallel with or after the data cache access.If physical addresses are stored in the prefetch unit 300, thetranslated portion of the physical address may be received from theTLBs, for example.

It is noted that the description of FIG. 3 above relates to one datacache port. There may be multiple data cache ports to support concurrentaccess. A similar discussion to that above may apply to each port. ThePLT memory 304 may include additional ports, for example, to supportconcurrent prefetch stream learning from each port.

Turning next to FIG. 4, a block diagram of an exemplary entry 320 thatmay be used in one embodiment of the PLT memory 304 is shown. The PLTmemory 304 may include multiple entries similar to entry 320, in anyarrangement (direct mapped, set associative, fully associative, etc.).

The entry 320 includes various data fields. The fields included in agiven embodiment depend on the type of prefetch pattern that theprefetch unit 300 is designed to support. For example, the entry 320 inFIG. 4 may support the detection of strided prefetch patterns. In FIG.4, the entry 320 includes a valid bit (V), a tag field (Tag), and threadID field (TID), a last miss address field (LMA) and a stride field(Stride).

The valid bit may indicate whether or not the entry is storing validdata. The PLT control circuit 302 may be configured to set the valid bitwhen writing the entry with data for a new potential prefetch stream tobe detected. Additionally, in some embodiments, the PLT control circuit302 may be configured to reset the valid bit in response to allocatingthe confirmed prefetch stream to one of the stream engines 306A-306N.Other embodiments may reverse the meanings of the set and clear statesof the valid bit, or use any other encoding.

The tag field may store the bits of the input address to the PLT memory304 that are not used to select an entry or entries in the memory, ifany. Thus, the tag field may be optional in some embodiments, e.g., inwhich all of the input address bits are used to select an entry. The TIDfield may store the thread ID for multi-threaded embodiments (and thusmay not be included in single-threaded embodiments). In single-threadedembodiments, a process identifier (PID) or a representation thereof(such as a hash of the PID) may be stored in the entry 320.

The last miss address field may store the most recently detected missaddress for the potential prefetch stream recorded by the entry 320. Thestride field may store the stride that is currently predicted to be thestride for the prefetch pattern that controls the predicted prefetchesforming the prefetch stream.

Turning now to FIG. 5, a flowchart is shown illustrating operation ofone embodiment of the PLT control circuit 302 in response to a loadoperation accessing the data cache 250. While the blocks are shown in aparticular order for ease of understanding, other orders may be used.Blocks may be performed in parallel in combinatorial logic circuitrywithin the PLT control circuit 302. Blocks, combinations of blocks,and/or the flowchart as a whole may be pipelined over multiple clockcycles. Thus, the PLT control circuit 302 may be configured to implementthe operation illustrated in FIG. 5.

The PLT control circuit 302 may receive the hit signal from the datacache 250 for the data address, along with the hit signal from the MRBand the hit signals from the stream engines. The hit signals may all bereceived in the same clock cycle, or may be received in different clockcycles as the load operation progresses through the L/S pipe in theprocessor. If the load operation is a cache miss (hit signal from thedata cache 250 deasserted—decision block 330, “yes” leg), a miss in theMRB (MRB hit signal deasserted—decision block 332, “yes” leg), and amiss in the prefetch streams in the stream engines (hit signals from thestream engines deasserted—decision block 334, “yes” leg), the PLTcontrol circuit 302 may enable a lookup in the PLT memory 304 (block336). A PLT memory 306 update may also occur. Further details of thelookup and possible update are provided below with regard to FIG. 6.

The hit determination for the data cache 250 and the MRB may be at acache block granularity against the cache blocks stored in the cache andthe miss requests stored in the MRB, respectively. For the streamengines 306A-306N, the comparison may be to the C Ptr in each streamengine, to indicate consumption of another prefetch in the prefetchstream. The granularity of the comparison may be the cache block.Alternatively, in other embodiments, the granularity of the comparisonmay be the same as the granularity of the stride detection (e.g. inembodiments in which the granularity of the stride detection is finerthan the cache block, as discussed above).

Turning now to FIG. 6, a flowchart is shown illustrating operation ofone embodiment of the PLT control circuit 302 to perform a PLT memory306 lookup (e.g. block 336 from FIG. 5). While the blocks are shown in aparticular order for ease of understanding, other orders may be used.Blocks may be performed in parallel in combinatorial logic circuitrywithin the PLT control circuit 302. Blocks, combinations of blocks,and/or the flowchart as a whole may be pipelined over multiple clockcycles. Thus, the PLT control circuit 302 may be configured to implementthe operation illustrated in FIG. 6.

In embodiments that implement a tag for a portion of the input addressto the PLT memory 304, the PLT control circuit 302 may compare the tagfrom the tag field of the selected entry (or entries) to the tag portionof the input address. Similarly, in embodiments that implement the TIDfield, the PLT control circuit 302 may compare the TID from the TIDfield of the selected entry (or entries) to the input TID.Alternatively, one or both fields of the entry may be CAM entries andthe comparison may be made in the memory 304. If the tag and/or TID donot match (decision block 340, “no” leg), the potential prefetch streamis a miss in the PLT memory 304. The PLT control circuit 302 mayallocate an entry to the potential prefetch stream (block 342) and maywrite the miss address to the last miss address field of the entry(block 344). The PLT control circuit 302 may also set the stride fieldto zero in the allocated entry, write the new tag and TID to therespective entry fields, and set the V bit in the entry.

In embodiments in which more than one entry is selected in response toan input address, the comparison of the tag and TID in each selectedentry may be performed in parallel. If the tag and TID match in any oneof the selected entries, the result of the decision block 340 may beyes. Additionally, the PLT control circuit 302 may select among theentries to allocate an entry using any desired algorithm. For example,the PLT control circuit 302 may attempt to select an invalid entry (Vbit clear). If all entries are valid, a random selection may be made, anLRU or pseudo-LRU algorithm may be used to select an entry, a U bitalgorithm may be used to select an entry, etc.

If the tag and TID match in an entry (decision block 340, “yes” leg),the PLT control circuit 302 may receive the contents of the matchingentry as a read output from the PLT memory 304. The PLT circuit 302 maycompute the stride as the difference between the current miss addressand the last miss address in the miss address field (block 346). If thestride field in the matching PLT entry is zero (decision block 348,“yes” leg), then the potential prefetch stream has not yet beenconfirmed. The PLT control circuit 302 may update the entry, writing themiss address to the last miss address field of the entry and thecomputed stride to the stride field of the entry (block 350). If thestride field in the matching PLT entry is non-zero (decision block 348,“no” leg), then a previous potential stride has been recorded for thepotential prefetch stream. If the currently calculated stride matchesthe stride field (decision block 352, “yes” leg), the prefetch streammay be confirmed. The PLT control circuit 302 may allocate the confirmedprefetch stream to a stream ending 306A-306N and may invalidate theentry in the PLT memory 304 (block 354). If the currently calculatedstride does not match the stride field (decision block 352, “yes” leg),the potential prefetch stream may not yet be confirmed. The PLT controlcircuit 302 may update the entry with the miss address as the last missaddress and the currently calculated stride as the stride (block 350).

The above embodiment confirms a prefetch stream after three loads havebeen detected in the stream. Other embodiments may implement schemes inwhich more than three loads are used to confirm a prefetch stream. Forexample, a match count field may be added to the PLT entries, and thestride in the entry may be matched multiple times to confirm a prefetchstream (e.g. counting the matches in the match count field, andcomparing the match count to a desired count).

It is noted that, in pipelined embodiments, it is possible that the nextPLT lookup to a given entry may start prior to the update of the givenentry from the prior lookup. Bypassing of data from the subsequentpipeline stages to the previous pipeline stages when the same entry isread may be used. Alternatively, the pipeline may be stalled in suchcases for the subsequent lookup until the previous update completes.

Turning now to FIG. 7, a flowchart is shown illustrating operation ofone embodiment of the stream engine 306A (and more particularly the SEcontrol circuit 312 for the embodiment shown in FIG. 3) and the prefetchcontrol circuit 308 in response to the allocation of a prefetch streamto the engine 306A. Other stream engines may be similar. While theblocks are shown in a particular order for ease of understanding, otherorders may be used. Blocks may be performed in parallel in combinatoriallogic circuitry within the stream engine 306A. Blocks, combinations ofblocks, and/or the flowchart as a whole may be pipelined over multipleclock cycles. Thus, the stream engine 306A/SE control circuit 312 andthe prefetch control circuit 308 may be configured to implement theoperation illustrated in FIG. 7, as discussed below.

The stream engine 306A may determine if a prefetch is ready to issue(decision block 360). In some embodiments, a prefetch may be ready toissue if the number of prefetches outstanding is less than the limit. Inone embodiment, the limit may be changed as the number of prefetchesthat are consumed by demand accesses increases. If demand accesses areconsuming the prefetches, the confidence that the prefetch stream isaccurate and continuing to be used may increase. Particularly, if thenumber of consumed prefetches equals the limit, the limit may beincreased.

A specific embodiment that increases the limit may initialize the PLimit value to the initial limit for prefetches, and may increment the PLimit as prefetches are consumed. Each time the incremented P Limitvalue doubles, the limit may be doubled as well. For example, in anembodiment, the initial limit may be 4 prefetches. As prefetches areconsumed, the P Limit is incremented. Once 4 prefetches are consumed bydemand accesses, the P limit has increased to 8. The new limit of 8 maytake effect, and thus the limit has been doubled. Viewed in another way,once the number of prefetches at least equal to the limit has beenconsumed, the limit is increased. Particularly, the limit may bedoubled. There may also be a maximum limit (based on the size of the PLimit value) beyond which the limit may not be increased. That is, the PLimit value may saturate at a maximum value. Other embodiments mayincrease the limit at different rates and in different ways. Forexample, an embodiment may initialize the limit at 4 and increase by 4each time, or increase by two each time, etc.

If a prefetch is ready to issue (decision block 360, “yes” leg), thestream engine 306A may request a prefetch from the prefetch controlcircuit 308 (block 362). If the prefetch control circuit 308 grants therequest (decision block 364, “yes” leg), the stream engine 306A (andmore particularly the SE control circuit 312) may issue the prefetch tothe address stored as the P Ptr in the registers 314 through the mux310, and may increment the P CNT to indicate that another prefetch isoutstanding (block 366). Additionally, the stream engine 306A may updatethe P Ptr, adding the stride to generate the next prefetch address(block 368). If the prefetch request is not granted (decision block 364,“no” leg), the prefetch is not issued and thus no stream engine state isupdated.

The prefetch control unit 306 may determine whether or not to grant aprefetch request in a variety of fashions. If more than one streamengine requests a prefetch concurrently, the prefetch control unit 306may arbitrate the requests to select a winner. Any arbitration mechanismmay be used (e.g. round robin, least recently selected, etc.).Additionally, in embodiments which filter prefetches through the datacache 250, the prefetch control unit 306 may grant a prefetch when theL/S pipe status indicates that the prefetch may be inserted into the L/Spipe (e.g. that a given pipeline stage at which the insertion occurs isidle).

If a data address is provided from the data cache 250 (whether it is acache hit or not), the stream engine 306A may compare the C Ptr to thedata address (decision block 370). The comparison may be at a cacheblock granularity, or a finer granularity of the stride is determined ata finer granularity than the cache block. If the C Ptr is hit by thedata address (decision block 370, “yes” leg), and the P Cnt is zero(decision block 372, “yes” leg), then a demand access has consumed theprefetch that has not been issued yet. In such a case, the stream engine306A may update both the P Ptr and the C Ptr by the stride amount(blocks 374 and 376, respectively), and may increment the P Limit (block378). If the C Ptr is hit by the data address (decision block 370, “yes”leg), and the P Cnt is not zero (decision block 372, “no” leg), then ademand access has consumed a previously issued prefetch. The streamengine 306A may decrement the P Cnt (block 380), since one fewerprefetches are outstanding, and may update the C Ptr with the strideamount (block 376). The P limit may also be incremented since anotherprefetch has been consumed (block 378).

Turning now to FIG. 8, a flowchart is shown illustrating operation ofone embodiment of the stream engine 306A (and more particularly the SEcontrol circuit 312) to determine if a prefetch is ready (block 360 inFIG. 7). Other stream engines may be similar. While the blocks are shownin a particular order for ease of understanding, other orders may beused. Blocks may be performed in parallel in combinatorial logiccircuitry within the stream engine 306A. Blocks, combinations of blocks,and/or the flowchart as a whole may be pipelined over multiple clockcycles. Thus, the stream engine 306A/SE control circuit 312 may beconfigured to implement the operation illustrated in FIG. 8.

The stream engine 306A may locate the most significant bit in the PLimit that is set (block 390). The stream engine 306A may test the bitin the same bit location of the P Cnt as the most significant set bit ofthe P Limit, and it if is clear (decision block 392, “yes” leg), then aprefetch is ready (block 394). If the bit in the same bit location ofthe P Cnt as the most significant set bit of the P limit is set(decision block 392, “no” leg), then a prefetch is not ready to beissued (block 396).

The above implementation, along with the incrementing of the P limitvalue for each consumed prefetch, may effectively double the limit eachtime the number of consumed prefetches equals the limit. For example, ifthe P limit is initialized to 4, bit 2 of the P limit (numbering theleast significant bit zero) is the most significant set bit. As long asthe P Cnt is less than 4, the limit has not been reached (and P Cnt bit2 is zero), so a prefetch may be ready. Additionally, as prefetches areconsumed, the P Limit is incremented. After 4 prefetches are consumed,the P Limit reaches 8 and the most significant set bit is bit 3. After 8more prefetches are consumed, the P limit reaches 16 and the mostsignificant set bit is bit 4. Accordingly, the increments of the P limitmay not affect the actual limit until the most significant bit toggles(i.e. at the next power of two).

In some embodiments, other factors may also affect whether or not aprefetch is ready. For example, in one embodiment, a prefetch stream mayterminate at a certain address limit. In one implementation, the addresslimit may be a 2 gigabyte boundary. Other embodiments may implement alarger or smaller address limit.

FIG. 9 is an example illustrating the operation of an embodiment of astream engine 306A-306N that uses the P Cnt and P Limit definitiondescribed with regard to FIG. 8. In FIG. 9, a table of actions is shown,with the order of the actions being the order in the table from top tobottom. The first action is an allocation of a prefetch stream to thestream engine. In the example, the first address is 60 (hexadecimal) andthe stride is 20 (hexadecimal). In addition to the action column, thetable includes columns for the C Ptr, P Ptr, P Cnt, and P Limit. Each ofthese columns are in hexadecimal format, and so omits the “0x” notation.In the first row, in response to the allocation, both the P Ptr and theC Ptr are initialized to 60. The P Cnt is 0 (no prefetches areoutstanding), and the P Limit is initialized to 4. Other embodiments mayuse other initial limits, or the limits may be programmable.

The next 4 actions in FIG. 9 are issued prefetches. Accordingly, the PPtr is updated by the stride amount each time and the P Cnt isincremented. After 4 issued prefetches, bit 2 of both the P Cnt and theP Limit (both 4 at this point in the example) are set, and no additionalprefetches are ready. The solid heavy line 400 indicates that the streamengine is idle.

Subsequently, a stream engine hit is detected for a demand access to thedata cache 250. The P Cnt is non-zero. Accordingly, the C Ptr is updatedby the stride amount (to 80) the P Cnt is decremented to 3, and the PLimit is incremented to 5. Since bit 2 of the P Limit is the mostsignificant set bit, and bit 2 of the P Cnt is 0, a prefetch is readyand is issued as the next action in the example, updating P Ptr to 100and the P Cnt to 4. At this point, even though the P Cnt is not equal tothe P Limit, the most significant set bit of the P Limit is still bit 2and bit 2 of the P Cnt is set. Effectively, the limit is still 4.Accordingly, the stream engine is again idle (solid heavy line 402).

The next five actions in the example are all stream engine hits.Accordingly, the C Ptr is updated by the stride amount each time, the PCnt is decremented, and the P Limit is incremented. After a total of 4stream engine hits (at entry 404), the P Limit has reached 8. The mostsignificant set bit is now bit 3, and thus the limit has increased to 8prefetches (or doubled). As additional stream buffer hits are detected,the P Limit continues to increment but the limit is still 8 until themost significant set bit changes again (not shown in the example of FIG.9). At the last entry in the table, a stream buffer hit occurs with theP Cnt=0. Consequently, the next prefetch that would be issued has justbeen consumed. Accordingly, both the C Ptr and P Ptr are updated by thestride amount (to address 120 in this example).

Exemplary System Embodiment

As described above, in some embodiments, processor 10 of FIG. 1 may beconfigured to interface with a number of external devices. Oneembodiment of a system including processor 10 is illustrated in FIG. 10.In the illustrated embodiment, system 800 includes an instance ofprocessor 10, shown as processor 10 a, that is coupled to a systemmemory 810, a peripheral storage device 820 and a boot device 830.System 800 is coupled to a network 840, which is in turn coupled toanother computer system 850. In some embodiments, system 800 may includemore than one instance of the devices shown. In various embodiments,system 800 may be configured as a rack-mountable server system, astandalone system, or in any other suitable form factor. In someembodiments, system 800 may be configured as a client system rather thana server system.

In some embodiments, system 800 may be configured as a multiprocessorsystem, in which processor 10 a may optionally be coupled to one or moreother instances of processor 10, shown in FIG. 10 as processor 10 b. Forexample, processors 10 a-b may be coupled to communicate via theirrespective coherent processor interfaces 140.

In various embodiments, system memory 810 may comprise any suitable typeof system memory as described above, such as FB-DIMM, DDR/DDR2/DDR3/DDR4SDRAM, or RDRAM®, for example. System memory 810 may include multiplediscrete banks of memory controlled by discrete memory interfaces inembodiments of processor 10 that provide multiple memory interfaces 130.Also, in some embodiments, system memory 810 may include multipledifferent types of memory.

Peripheral storage device 820, in various embodiments, may includesupport for magnetic, optical, or solid-state storage media such as harddrives, optical disks, nonvolatile RAM devices, etc. In someembodiments, peripheral storage device 820 may include more complexstorage devices such as disk arrays or storage area networks (SANs),which may be coupled to processor 10 via a standard Small ComputerSystem Interface (SCSI), a Fibre Channel interface, a Firewire® (IEEE1394) interface, or another suitable interface. Additionally, it iscontemplated that in other embodiments, any other suitable peripheraldevices may be coupled to processor 10, such as multimedia devices,graphics/display devices, standard input/output devices, etc. In oneembodiment, peripheral storage device 820 may be coupled to processor 10via peripheral interface(s) 150 of FIG. 1.

As described previously, in one embodiment boot device 830 may include adevice such as an FPGA or ASIC configured to coordinate initializationand boot of processor 10, such as from a power-on reset state.Additionally, in some embodiments boot device 830 may include asecondary computer system configured to allow access to administrativefunctions such as debug or test modes of processor 10.

Network 840 may include any suitable devices, media and/or protocol forinterconnecting computer systems, such as wired or wireless Ethernet,for example. In various embodiments, network 840 may include local areanetworks (LANs), wide area networks (WANs), telecommunication networks,or other suitable types of networks. In some embodiments, computersystem 850 may be similar to or identical in configuration toillustrated system 800, whereas in other embodiments, computer system850 may be substantially differently configured. For example, computersystem 850 may be a server system, a processor-based client system, astateless “thin” client system, a mobile device, etc. In someembodiments, processor 10 may be configured to communicate with network840 via network interface(s) 160 of FIG. 1.

Numerous variations and modifications will become apparent to thoseskilled in the art once the above disclosure is fully appreciated. It isintended that the following claims be interpreted to embrace all suchvariations and modifications.

1. A stream engine comprising: one or more registers configured tostore: a first pointer identifying a next expected demand accesscorresponding to a prefetch stream that is assigned to the streamengine; a second pointer identifying a next prefetch in the prefetchstream; and limit data indicating a limit to a number of prefetches thatare permitted beyond the first pointer; and a control circuit coupled tothe one or more registers and configured to generate prefetchesresponsive to the limit data, and wherein, responsive to demand accessesconsuming at least a number of prefetches equal to the limit, thecontrol circuit is configured to increase the limit.
 2. The streamengine as recited in claim 1 wherein the limit data includes a firstcounter indicative of a number of prefetches that have been issued andnot yet consumed, and wherein the limit data further includes a secondvalue that is initialized to the limit.
 3. The stream engine as recitedin claim 2 wherein the control circuit is configured to issue a prefetchusing the second pointer, and wherein the control circuit is configuredto increment the first counter.
 4. The stream engine as recited in claim3 wherein the control circuit is configured to update the second pointerto the next prefetch address in the prefetch stream.
 5. The streamengine as recited in claim 4 wherein the prefetch stream is strided, andwherein the control circuit is configured to update the second pointerby adding the stride.
 6. The stream engine as recited in claim 2 whereinthe control circuit is configured to detect a consumption of the data atthe first pointer, and wherein the control circuit is configured todecrement the first counter and to increment the second value inresponse.
 7. The stream engine as recited in claim 6 wherein the controlcircuit is further configured to update the first pointer to indicatethe next expected demand access after the consumption.
 8. The streamengine as recited in claim 2 wherein the control circuit is configuredto determine that a prefetch is ready to be issued responsive todetecting a clear bit in a bit position of the first counter that is asame bit position as a most significant set bit in the second value. 9.The stream engine as recited in claim 2 wherein, responsive to aconsumption of data at the first pointer and the first counter beingequal to zero, the control circuit is configured to increment the secondvalue and update both the first pointer and the second pointer to a nextprefetch address in the prefetch stream.
 10. A processor comprising: adata cache; and a prefetch unit coupled to the data cache, wherein theprefetch unit is configured to identify a prefetch stream in cachemisses from the data cache, and wherein the prefetch unit is configuredto issue prefetches predicted by the prefetch stream to prefetch datainto the data cache, and wherein the prefetch unit is configured tomaintain limit data that indicates a number of prefetches that arepermitted to be outstanding beyond a most recent demand access, andwherein the prefetch unit is configured to increase the limit responsiveto the number of demand accesses that consume prefetched data at leastequaling the limit.
 11. The processor as recited in claim 10 wherein thelimit data includes a first counter indicative of a number of prefetchesthat have been issued and not yet consumed, and wherein the limit datafurther includes a second value that is initialized to the limit. 12.The processor as recited in claim 11 wherein the prefetch unit isconfigured to issue a prefetch and increment the first counter.
 13. Theprocessor as recited in claim 12 wherein the prefetch unit is configuredto detect a consumption of prefetch data and to decrement the firstcounter and to increment the second value in response.
 14. The processoras recited in claim 12 wherein the prefetch unit is configured todetermine that a prefetch is ready to be issued responsive to detectinga clear bit in a bit position of the first counter that is a same bitposition as a most significant set bit in the second value.
 15. Theprocessor as recited in claim 12 wherein, responsive to a consumption ofdata in the prefetch stream and the first counter being equal to zero,the prefetch is configured to increment the second value.
 16. A methodcomprising: a prefetch unit identifying a prefetch stream in cachemisses from a data cache; the prefetch unit issuing prefetches predictedby the prefetch stream to prefetch data into the data cache; theprefetch unit maintaining limit data that indicates a number ofprefetches that are permitted to be outstanding beyond a most recentdemand access; and the prefetch unit increasing the limit responsive toa number of demand accesses that consume prefetched data at leastequaling the limit indicated by the limit data.
 17. The method asrecited in claim 16 wherein the limit data includes a first counterindicative of a number of prefetches that have been issued and not yetconsumed, and wherein the limit data further includes a second value,and the method further comprising initializing the second value to thelimit.
 18. The method as recited in claim 16 further comprising theprefetch unit issuing a prefetch and incrementing the first counter. 19.The method as recited in claim 17 further comprising: the prefetch unitdetecting a consumption of prefetch data; the prefetch unit decrementingthe first counter in response to the detecting; and the prefetch unitincrementing the second value in response to the detecting.
 20. Themethod as recited in claim 17 further comprising the prefetch unitdetermining that a prefetch is ready to be issued responsive todetecting a clear bit in a bit position of the first counter that is asame bit position as a most significant set bit in the second value.